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Abstract — This paper reports an effort to consolidate numerous 
coherence-based sparse signal recovery results available in the 
literature. We present a single theory that applies to general 
Hilbert spaces with the sparsity of a signal defined as the number 
of (possibly infinite-dimensional) subspaces participating in the 
signal's representation. Our general results recover uncertainty 
relations and coherence-based recovery thresholds for sparse 
signals, block-sparse signals, multi-band signals, signals in shift- 
invariant spaces, and signals in finite unions of (possibly infinite- 
dimensional) subspaces. Moreover, we improve upon and gener- 
alize several of the existing results and, in many cases, we find 
shortened and simplified proofs. 

I. Introduction 

The sparse signal recovery literature is vast and has evolved 
along several threads with recent focus mostly on probabilistic 
results. This paper constitutes an attempt to consolidate the 
numerous coherence-based recovery results available in the 
literature. More specifically, we formulate a single theory 
that applies to finite- and infinite-dimensional Hilbert spaces, 
in combination with sparsity defined as the (finite) number 
of (possibly infinite-dimensional) subspaces participating in 
a signal's representation. The general coherence-based re- 
covery thresholds we find contain the known thresholds in 
the following settings as special cases: (i) sparse signals 
in finite-dimensional spaces lffl~ll4l, (ii) block-sparse signals 
0, @, (iii) multi-band signals J7J— (5), (iv) signals in shift- 
invariant spaces iflOl . and (v) signals in finite unions of finite 
or infinite-dimensional subspaces lfTTI - lfT3l . In addition, we 
improve upon the thresholds in [5| and we generalize the 
uncertainty relation in [10]. We introduce suitable generaliza- 
tions of PO-minimization |2|, basis pursuit [2|, and orthogonal 
matching pursuit fl4l . Finally, we indicate how the results on 
signal separation reported in IflD . ITT61 can be extended to the 
general Hilbert space setting considered here. 

Key to our results are definitions of coherence [2] and 
mutual coherence J3], |[T6l that work for our general setting. 
Based on these definitions, we obtain a general kernel uncer- 
tainty relation which is then used to establish general recovery 
thresholds. Similarly our definition of mutual coherence paves 
the way to a general uncertainty relation that yields fundamen- 
tal limits on how sparse a signal in a general Hilbert space 
can be under two different representations. All theorems in 
this paper are given without proof. 

Notation: Lowercase boldface letters stand for column 
vectors and uppercase boldface letters designate matrices. For 
a vector a, the fcth element is written a^. For the matrix A, 
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A H is its conjugate transpose, its fcth column is written a,k, 
and the entry in the fcth row and ith column is denoted by 
The spectral norm of A is || A|| 9 2 , fT m ; n (A) and cr max (A) are 
the minimum and maximum singular value of A, respectively, 
and ^ are Hilbert spaces equipped with the norm 
and ||-||gp, respectively, and ffl has direct sum decom- 
position 02] Ch. 5.20] = 0" =1 ,ye « where n < oo. 
We define «W to be the canonical projection of v onto 



JT<». For v G Jif, 



{i : > 0}| and 



and 



,(S) 



I ,"/6 



. We define (s) = se5 Jf^ 



to be the projection of v onto Jif^- S \ We say 
that a signal v G is £5 -concentrated to the set S if 

ll w(,S) IU,i > i 1 - £ <s) IMIje-,1. where < e 5 < 1. We 
define G to be the all zero vector with a one in 
the ith position. For an operator cp; J/f — > with ad- 
joint ip H , uj min (ip) = 'mf ve M>\\ip(v)\\^/\\v\\ M ,, uj max (ip) = 

sup^g^. ||v?( u )ll^/IHjr- an d k er(^) = {« G ■ <p(v) = 
0}. For a € 1, we set [a] + = max{0, a}. The cardinality of 
a set S is denoted as <S|. The Fourier transform operator is 
written T. 

II. Signal and Sampling Model 

Let 3? and be Hilbert spaces, with dimensions N and M, 
respectively, possibly infinite. Assume that ffi 1 = Jf?( l \ 
n < 00, and set dj = dim(^W). We describe the sampling 
of signals in through the application of a bounded linear 
operator $ : M' — > ', which we call a sampling operator. With 
<£> we associate the operators (pi : M'^ — > Sf, for i = 1, n, 
obtained by restricting the action of <£> to the subspace . 
It follows from the linearity of $ that $(u) = Y^i=i i Pi( v ^)- 
We require that each <pi be injective. 

For N = Y2i=i di < °°> tne action of $ can be represented 
through a matrix D £ C MxN according to <l>(v) = D«, 
v G C N . Taking D i = [ di 1 ■ ■ ■ di d ] to be the set of columns 
of D that correspond to we have ifii(v^) — D iv^\ 

for i = 1, n. 

III. Definitions of Coherence 

Key to our results are definitions of coherence, mutual 
coherence, and spark for general sampling operators. 

Definition 1 (Hilbert space coherence): Let 34? and be 
Hilbert spaces and let $: Jff — » be a sampling operator. 
We define the Hilbert space coherence of <£>a^j 



max ■ 



(1) 



'By assumption the operators ipi are injective, hence u) m \ n (<fi) > 0. 



We can interpret /ijf ($) as a measure of closeness of the 
subspaces Jfw under the action of <I>. 

Definition 2 (Mutual Hilbert space coherence): Let Jffi, 
J?2, and S? be Hilbert spaces and let $: M[ Sf and 
"J : ^2 — > ^ be sampling operators. We define the mutual 
Hilbert space coherence of $ and ^as 



u 

max 

M ^min 



(2) 



The mutual Hilbert space coherence extends the definition 
of mutual coherence in 0, (16). The setting of 0, IfTBI 
is recovered as follows. Let M\ — C^ 1 , = C* 2 , and 
eg _ j-<a/ Represent the sampling operators $: ^ — > ^ and 
*f>: 3%2 ~ > & by the matrices A and B, respectively, so that 
= Ai> and ^(m) = Bit. Then, we have 



max ■ 



= max|(aj, 6j) 
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where fj, m is the mutual coherence as specified in [16|, and (a) 
follows since in IfTBI A and B are assumed to have columns 
with unit £2 -norm. 

We will also need a general definition of spark [4|, [18|. 

Definition 3 (Hilbert space spark): Let Jif and Sf be 
Hilbert spaces and let Jf — > Sf be a sampling operator. 
Then 



spark(<I>) = min \\v\ 

t)6kor($)\{0} 



(3) 



The spark of a sampling operator is the smallest number of 
subspaces that a non-zero signal v £ J$? in ker($) can occupy. 

IV. Recovery Thresholds 

With our general definitions of coherence and spark, the 
general recovery thresholds below follow without difficulties. 
We start with a general kernel uncertainty relation. 

Theorem 1 (Kernel uncertainty relation): Let $: — > & 
be a sampling operator with Hilbert space coherence 
Let v £ J4? be ss -concentrated to S. If = 0, then 



\S\ > (l-e s ) ll + frjeW) 



(4) 



We next define two optimization problems for the recovery 
of a signal v £ from its measurements z = $(t?) £ Sf. 
The first one, Jf?-P0, aims to find the signal that explains the 
given measurements while occupying the fewest subspaces: 



(Jt?-P0) minimize IHIj^q subject to $(«) = z. 



(5) 



Furthermore, we consider a modified version of basis pursuit: 
(J/f-BP) minimize \\v\\ ^ 1 subject to <£>(?)) = z. (6) 

Recovery thresholds for J$?-P0 and ,yf-BP can now be 
derived from the kernel uncertainty relation in Theorem [TJ 

Theorem 2: If v £ J$? satisfies $(u) = z and 

\\v\\^ a < spark($)/2, (7) 
then v is the unique minimizer of J$?-P0 applied to z. 



In addition, we have the following bound, spark($) ^ 1 + 
which combined with Theorem [2] allows us to 
conclude that J^-PQ returns the correct solution if 



(8) 



We next provide a recovery condition for J%?-BP. 

Theorem 3: If v £ satisfies = z and © holds, 

then J^-BP applied to z returns the correct solution v. 

A commonly used alternative to BP is orthogonal matching 
pursuit (OMP) lfT4l . fl9l . We next present a Hilbert-space ver- 
sion of OMP, which we call if-OMP. This algorithm works 
by iteratively identifying the subspaces J^W participating in 
the representation of v and computes an approximation to v, 
denoted as Uj, in the ith iteration. The corresponding residual 
in the ith iteration is given by r*j = z— $(i>i). The algorithm is 
initialized with ro f- z and £ ■<— 1, and performs the following 
steps until ||r.;||^ = 0: 

1) Find 



arg max 



of participating sub- 



2) Update the list 
spaces: 5, <— <Sj_x U {£}. 

3) Find the best approximation to v with support 5*: 



Vi ^— arg mm \\z - 



4) Update the residual and £: <— z — £ •<— £ + 1. 

Theorem 4: Let $ : Jff -> ^ be a sampling operator. Then 
-OMP applied to z — $(1?) returns the correct solution v 
if © is satisfied and will require exactly ||w||^ iterations. 

Note that implementing the algorithms mentioned above, 
when is infinite-dimensional, is non-trivial. Some alter- 
natives to Jt°-BP and Jf-OMP, such as SBR2/4, have been 
proposed for blind multi-band sampling [9|, which is a special 
case of our setup. It is an interesting open problem to extend 
these algorithms to the general framework in this paper. 

V. Discussion of Recovery Thresholds 

We next show how the recovery thresholds in |Q]-[|5], 10- 
(2)> ifTTl follow from the general recovery threshold (|8). The 
results in [6|, which pertain to a generalization of [5 1 allowing 
for different subspace dimensions, can be recovered following 
the same methodology, but this will not be detailed here due 
to space constraints. 

A. Sparse signal recovery 

The (coherence-based) thresholds in [1|-[4| are recovered 
as follows. Set Jif = C N and <S = C M . Take the sampling 
operator $ to be represented by the matrix D £ C MxN , with 
unit ^2-norm columns di. Take J#°M to be the 1-dimensional 
subspace spanned by ej £ C N , so that N = n. The action of 
ifi : — s> <$ is represented by ipi(v^) = diV^ — djUj. 
Since uj min (ipi) = ||dj|| 2 = 1 and uw^f Vi) = \( d i, d j)l 
we get = max^j \(di, dj)\, which is exactly the definition 
of coherence as introduced in 0-lffl. The recovery threshold 
© for JT-PO, Jf-BP, and JT-OMP (which then reduce 
to PO, BP, and OMP, respectively) is thus equal to the 



corresponding thresholds in ||2]-(|4]. As an aside the general 
result (0 shows how dictionaries with unnormalized columns 
should be treated, specifically what the appropriate measure 
of coherence is, and what the selection criterion in Step [1] of 
(jr-)OMP should be. 

B. Block- sparsity 

The results for the block-sparse setting considered in ||5] are 
recovered as follows. Set = C N , <S = C M , and N = nd, 
where d is the block size and n is the number of blocks (and 
hence the number of subspaces JtfW). As before, the sampling 
operator $ is represented by the matrix D G C Mx,v with 
unit <?2-norm columns. Let Jg°W be the subspace spanned by 
{e(i-i) d +i, — , e id }, and set D i = [d^ 1)d+1 ■■■ d id ], so 
that (fii(v^) = D iv^K From ([TJ the Hilbert space coherence 
is 



max ■ 



(D ifDj 



,(D i) 



(9) 



We next show how the recovery threshold ((8) improves 
upon that reported in J5] Thms. 2 and 3], which states that 
recovery using (L-OPT) [5. Eq. 32] and BOMP [5, Sec. IV- 
A] (our J^-BP and Jf-OMP, respectively), is successful if 



< (l + /T 1 )/2. Here 



li = 



Mb = Mb(D) = max 



^b(D) 
1 - (d- l)v 

,((D if!) 



max 



max 



(D 



D £, 



and D £j is the ith column of D & The following steps 
establish that [i^> ^ fi, thereby proving our claimP] 



min fe er min ((D fc) H D 
((D tfDj 



(a) ^max \ 

^ max 

i,j,j¥i [1 — (rf — 1)^]' 



where we applied the Gersgorin disc theorem [20. Th. 6.1.1] 
in (a). When (D i) H T) i = I d , for all i, we have = (i, 
but one can easily find examples where the strict inequality 
fijif < p, holds. 

C. Multi-band signals 

We next show how our results apply to sparse multi-band 
signals as considered in (8), (9), ||2T], (22). Let be the 
space of functions band-limited to the interval [0, 1/T) and 
for a signal v G ,j*t? , let V be its Fourier transform. For 

2 It is possible that fi < and since fi ^ ("J?) is always non-negative, we do 
not have /i (4>) ^ fj, in this case. However, in this instance [5. Thms. 2 and 
3] say that we cannot guarantee the recovery of any signal, but the right-hand 
side of (8) is positive, thus trivially improving upon the recovery thresholds 
in (5] Thms. 2 and 3]. 



simplicity of exposition, assume that the interval [0, 1/T), 
is divided into n disjoint intervals Xi,...,l n , with li = 
[(i ~ 1) / (nT) , i / (nT)) , i = l,...,n. Define the subspaces 
JTW = {v e L 2 (K) : V(f) = 0, for all / g l/\. Thus, for a 
signal v G Jff, the sparsity level ||u||^ * s me num ber of 
frequency bands 1 L occupied by V . 

We next demonstrate how the multi-coset sampling scheme 
of Q, O can be analyzed in our framework. Multi-coset 
sampling maps the signal v to m ^ n sequences z 



as 



follows: 



,{*) 



= v(tnT + kT), k = l,...,m, £ E Z. 



To obtain an explicit characterization of the corresponding 
sampling operator $ we will work in the frequency domain. 
The Fourier transform of is given by 



Z<«(/) 



( 



nT / 

2-Kiki/n 



£am^(/), 



where A M = (nT)' 1 exp(2mkl/n) and V^(f) = 
V(f + £/(nT)). Then, the action of the sampling operator, 
$ : 34? — > Sf , can be represented in terms of the continuously 
parametrized linear system of equations 

Ai,i A1.2 





for / G [0,1/ (nT)), We have thus established a finite- 
dimensional continuously indexed matrix representation of $ 
IfTTl . Based on this insight, we next show that 



and 



spark($) = spark(A) = m, 



(11) 



(12) 



which means that we can reduce the computation of Hilbert 
space spark and Hilbert space coherence of an infinite- 
dimensional operator to that of a finite matrix that does not 
depend on /. Since (lOj holds for all / G [0, l/(nT)), for v 
to lie in the kernel of $, V(f) must be in ker(A) for each 
/ G [0,1/ (nT)). One can then show that this implies that 
spark($) = spark(A). The second equality in (fTTT i follows 
since A consists of the first m rows of the n x n DFT matrix 
and hence spark(A) = m lETI . 

To prove ( TTZb . note that for u G with Fourier trans- 

form U, (fi : 34?^ — > & is given by the matrix representation 



/Ai, 



and has adjoint 



V>?(X)(f) 



where 




(/)= E a m iW (/)■ |W||V| ^ 



\k=l 



X 



Hence, for u G with Fourier transform U, we have 



2J X* tj Xi,iU 

u 



i=l 

AfA 



E* 



|t/| 



(13) 



where Xj is the jth column of A. Since (13[ holds for all U, 
it follows that 



Wmm( c Pj <Pt) =W max (y! J . ipi) = |A^A^| 



and hence 



\XfX e 



3,1,3^1 Umm{Vi<Pl) \\X e \\t 



A*^r(A). 



From J7), (9] we know that to recover a multi-band 
signal with bandwidth s/(nT) (and with unknown spec- 
tral occupancy), it is necessary to sample at a rate f s = 
m/(nT) ^ 2s/(nT). Theorem [2] implies that uniqueness of 
J^-PO recovery is guaranteed for multi-band coset sampling 
if spark($)/2 = m/2 > s. Hence, sampling at rate at least 
2s/(nT) is also sufficient to recover an s-sparse signal and 
recovery of the (multi-coset sampled) signal can be achieved 
through JT-PO. 

D. Relation to further results 

Theorem [2] in this paper implies ifTTl Prop. 4] and ifTTI 
Eq. (23)] with the observation that the generalized Gram 
matrix in IfTTl Eq. (17)] plays the role of the sampling 
operator <£> in our framework. Our Theorem |2] also implies 
(111 Th. 2.2]. 

VI. Uncertainty Relations and Signal Separation 

Another thrust in the sparse signal recovery literature deals 
with the recovery of sparsely corrupted signals [16]. The main 
tool underlying this line of work is an uncertainty relation that 
sets a limit on how sparsely a given signal can be represented 
concurrently in two different dictionaries JT], ITT51 . [16]. We 
next formulate a Hilbert space version of this uncertainty 
relation, which is then used to recover and generalize results 
in (TO) and H6). 

Theorem 5 (Uncertainty relation): Let M'x^M'i, and §f be 
Hilbert spaces . and let $ : J#[ -> 5f and * : M2 -t & be 
sampling operators. Let u G M\ and v G J&2 be signals that 



are en- and £y-concentrated to the sets U and V, respectively, 
and assume that = ^(v). Then, we have 

1 



[(1 - 6k) (1 + -|W| 



x [(1 - e v ) (1 + useW) - \V\ ^^>)\ + 



(14) 



Remark: |[T6l Th. 1] can be recovered from Theorem |5] by 
noting that $ and ^ play the role of the dictionaries A and 
B, respectively, as used in |[T6l . Then = ^(v) becomes 
An = Bv and |T6l Th. 1] follows since = (i m , 

AUr($) = Ma, and fijt(^) = U&, with fi m ,fi a , and fi b as 
defined in ffiA . 

A. Shift-invariant spaces 

We next show how Theorem can be used to recover ifTOl 
Th. 1]. Consider the shift-invariant space 



6^ z:z(t)= v i ^Mt-kT)^^ eh, Vi>, 



i— l,...,ni 



(15) 



with Tii generators fa G La(M) and \\fa\\ 2 — L for all i. Set 
J^i to be the space of vector sequences 



/ V (D 



(16) 



,(™i) 



with «W g for all i. Define the operator i/3j : £2 — > ©0 by 



with adjoint iff: — >• ^2 given by cpi*[z) 
{(z(-),fa(- — £T))} ieZ . The sampling operatoi0 $: 
6^ is then given by 

ni ni 

*(«)=EE w ^(--* T ) = E^( w(i) )- 



(18) 



i=i fee2 



i=i 



A signal t> G Jif is s-sparse if at most s of the sequences «W in 
( fT8l are non-zero, i.e., if \\v\\^e ^ s, and in the terminology 



of [10|, 



\3H? 



q is the number of active generators. 



Now let us consider a set of n-x generators 0, G L-2(R) 
where ||0j|| 2 = 1, for all i, and the space 



G e = \z: z(t)= v^6i(t-kT),v^ e£ 2 , Vi 

i=l,...,n 2 
fe£Z 

Let ^2 be the space of vector sequences, as in ( TTSI l. but with 
m replaced by 77,2, and define the operators di \ £2 — > &e 
and O : M2 — > &g as in ( fTTb and ( TT8l , respectively, with ^ 
replaced by Qi. Suppose that z = <5>(i>) = Q(u). We now 
establish a limit on the sparsity of u and i>. 

3 In this case, the sampling operator rather behaves like an interpolation 
operator as it maps a sequence to a continuous-time signal, but to maintain 
consistency with the rest of the paper we still refer to it as a sampling operator. 



Following [ 1 1 the generators will be assumed to satisfy: 

(<M--fcT),^(--£T)) = {6 i (--kT),6 j (--£T)) 

J 1 if i = j and k = I 
) otherwise. 



|t/ 1 )|| 2 , for all i, and for all «W g £ 2) 



Then ||<Pi(«W)|| 2 

hence comin(fi) — 1, for all i, and similarly w m i n (#f) = 1, for 
all £. For i ^ j and v^' £ £2, we have 

^(^))| 2 =^|(^-(^)) ) ^(.-feT)) 



E 

fcez 

E| 

fcez 



£>f <p j (--£T),<j> i (--kT) 



,(J)\ 



|<^-(--£T),<M--fcT))| 2 = 0, 



=0 



and similarly ||i?f d 3 (u u) ) || 2 = 0, for all u^) G £ 2 - 
Therefore, ^^(<1>) = max^ ; w max (<^y>j) = ar, d similarly 
= 0. This gives 



At^($, 6) = max ■ 



c(W?*i) 



mapcuj ma (ipf<&j). 



l i3 ^niin 

The uncertainty relation (fl4l > hence reduces to 

INU,oll«Hjr,oM3e-(* J e)>l. (19) 

where we assume perfect concentration (since this is the case 
considered in [10]), i.e., eu = £\> = 0. We now show that 
(IT9b is the uncertainty relation in ifTOl Th. 1], which, in our 
notation, is given by ( fT9b but with 9) replaced by 

0) = max ess supli?^ g (e^) I , 

where 

R^,e r (^) = F{{(0r(-),M- ~ m)} kez }(^) ■ 

It therefore suffices to prove that Wmax (^f^r) = 
ess sup fe [ 0) 2 W )|iZ^ 4 ,e r (e^) |. For u (r) G ^2, we have 




(•-kT),&(.-mT)) 



where A m>fe = (9 r (- - kT), <f> t (- - mT)). Then 



!jj n 



sup 

IK'll^ 1 

sup 

HvWll =1 



Av {r) 



2 

= II vs. I 



2^2 



Since A is a doubly infinite Toeplitz matrix, its operator 



norm I A | 



is given by the essential supremum of the 



have w max (cpff i? r ) = ess sup^ € [ 2 ^ \ Rcj> e ,e r (e^)\, which con- 
cludes the proof. 

We finally note that our Theorem [5] also applies to 
nonorthogonal generator sets {<fii} and {9j} with potentially 
different shift parameters, thereby extending the uncertainty 
relation in [ 10 1 . 
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Fourier transform of a row of A 1231 p. 62]. We therefore 



